Formal Derivation of Mesh Neural Networks with Their Forward-Only Gradient Propagation

This paper proposes the Mesh Neural Network (MNN), a novel architecture which allows neurons to be connected in any topology, efficiently route information. In MNNs, information is propagated between throughout state transition function. State and error gradients are then directly computed from updates without backward computation. The MNN propagation schema formalized derived tensor algebra. proposed computational model can fully supply gradient descent process, potentially suitable for very large scale sparse NNs, due its expressivity training efficiency, with respect NNs based on back-propagation graphs.